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ABSTRACT 

The first year of an individualized bilingual magnet 
project was evaluated. A total of 254 elementary ahd_ secondary 
students in Philadelphia participated. Multimedia bilingual educat ion 
was provided to students of hoh-Eiglish speaking backgrounds at two 
sites. At the elementary site, bilingual. educat ion was of fered in 
Spanish, Korean , Vietnamese > French, Italian^ and Chinese. Evaluation 
of pupil performance objectives suggested that^ the magnet pupils 
acquired English more rapidly than did similar pupils city-wide. 
Dropout incidence in the high school group was very low, but no 
significant change in attendance was noted. (Author/RW) 
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The bilingual project, An Individualized Multi-media. Bilingual Magnet Model 
was funded by the U.S. Office of Education several months later than 
anticipated. As a result, the completion of some enabling objectives was delayed. 
Other enabling objectives were modified. By January, the program was in 
full operation and by year-end the program was serving 254 pupils . *At the 
elementary site , bilingual education was provided in Spanish, Portuguese 
and Korean. At the secondary site instruction was provided in Spanish, Korean, 
Vietnamese, French, Italian and Chinese. Evaluation of pupil performance 
objectives suggested that pupils acquired English more rapdily than did similar 
pupils city-wide. Dropout incidence in the high school component was very 
low . Year-end data were also gathered about reading in Enj£ish and Spanish 
and high -school pupils' attendance. These descriptive data are expected to 
be useful in evaluation* of 'the first full year of program operation, 1977-1978. 
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OBJECTIVES FOR 1976-1977 

— - ^ 
Enabling Objectives 

1. By the beginning of the 1976-1977 school year, the resource specialists 
arid teacher aides speaking the languages required in each of the two 
program sites will be employed. Other st,aff with required expertise 
will be. employed. 

« 

2. From September 1976 through December 19 76 /the Project Director, the 
program staff and the Director of Computer-based Instructional Systems 
will purchase and'have installed ike necessary instructional materials 
and equipment for the Bilingual Individualized Multi-Media Learning 
Center. This will include: 

Acquiatibh of appropriate grade-level instructional materials 
for teaming reading and for other subjects -in each of the 
languages of the program. 

Organization of the Bilingual Individualized Multimedia 
Learning Center at each of the schools served-by the pirogram. 

4 • m _ _ _» __*_ " __ _ _ 

Reproduction and cMiveiy of off-line materials needed for the 

computer- managed and computer- assisted instruction at the 

secondary school site. 

Acquisition of other programmed instructional materials. 

_ . 

,3. A program will begin in September, 19 76 that will attract non-English-, 
dominant Spanish-, Portuguese-^ Korean-, Chinese- , Vitenamese-', 
Italian - and French-- speaking pupils who live out side the regular 
boundary of the project schools , thus creating the "magnet" programs. 
: The recriiitment program will include: 

News releases sent to newspapers serving each of the language 
groups. 

Ann ouncements on radio programs serving speakers of the 
languages of the program. ^ 

Pupil recruitment by members of the program's parent advisory 
group . 



Notification to all principals of elementary and secondary schools 
describing the new programs and asking them to identify 
students who can benefit from participation in them. 



4. Program staff speaking Cjjhinese, Italian, Portuguese, Vietnamese, 
Korean and French will, with the as sistance of supervisors and 
evaluation personnel , develop informal: reading inventories tbbe 
^used in subsequent yeais to evaluate mother-tongue reading per- 
formance of pupils*s peaking these languages . These inventoried 
will be prepared'by June, 1977. 

P rpduct Objectives s 

v . 

1 . The fate of acquisition of En glish speaking and comprehension skills of 
pupils, as measured by the Philadelphia ESOL Screening Test and the^est 
of Aural Comprehension, will be increased to a statistically significant 
degree (p <.05) . 



2 . Th^ rate of acquisition of English reading skills , as measured by the Wide 
Range Reading Test, Will increase significantly (p<.05) among ,pupils 
enrolled in Grade 2 and above. : 

The rate of acquisition, of Hispanic pupils 1 reading competence in the 
Spanish language will increase significantly (p <.0 5) . The tests to be 
used, depending on grade level, are: 

Test de destrezas basicas en lecutra , 
Prueba de lectura (de Puerto Rico) and . 
The Inter- American Tests of Reading (Level 4) . 



4. Fewer pupils served by the program in 1976-1977 will drop out of high . 
school than did pupils of the same ethnic origin^who were attending the 

. same school in 1975-1976. Level of statistical significance will be p< .05. 

5. Secondary level pupils served by the program, who attended school in 
Philadelphia in 19 75- 1976, will have significantly (p< .05) fewer absences 
in the program than they did the year before the program began. 



PREFACE 

» 

The first annual evaluation of this project consists of three parts . The 
first two are status reports assessing the implementation of the program. The 
third part, the final report, emphas izes evaluatio n of ''product" or pupil per- 
formance objectives. In addition, o ne appendix, discussing city -wide 
performance of Title I p upils in the ESOL program completes the report. The 
appendix is referred to in the evaluation of the first product objective. 



STATUS REPORT 



October 1, 19 76 — December 30 , 1976 

This report describes program status as of late December, 1976. 

Pinal approval and official notification of funding were expected 
during July 19 76 , but were received on October i, 1976. Because of the late 
date of the funding, preliminary program activities which should have* taken place 
in fee summer of 1976 were not implemented until afterOctbber 1. As a result, 
completion of enabling object! yes Was delayed and the sequence of comple- 
tion dates was- modified. Product objectives were also modified to reflect 
the late funding. 

At South Philadelphia High School and Bi rney Elementary School, teach- 
ing staff were available for instruction . 'At the high school, a French-, an 
Italian- , two Spanish-^ and a Vietnamese- and Chinese speaking teacher already 
in service became members of the program faculty . A Korean-speakin g teacher 
was hired. All of these teaching positions are funded by tile LEA. A Spanish- 
speaking aide was hired to manage the high school Individualized Multi-Media 
Center. At the elementary school site, two Korean- , two Spanish- and & 
Portugese-speaking teacher already in service became members of the program 
faculty. These teaching positions were funded by the LEA. An aide, who is a 
native speaker of Portugese, was hired to manage the Individualized Multi-Media 
Center at the elementary school Examinations for two Site Coordinator posi- 
tions were held and a list of eligible candidates prepared. The Site Coordinator 
positions were filled December 20, 1976. 

V 

y The Multi-Media Center at the high school is to provide comp u te r-b as e d 
instruction. As of December, four computer terminals were installed in a class- 
room. Soft-ware installation and connection to a time- shared computer were pro- 
gressing, but not completed. At both sites, some instructional materials and 
audiovisual equipment used in the past were on hand, some new purchases for 
the Multi -Media Centers were delivered, and other materials and equipment were 
on order. At the high school the center was in use by December; at the elementary 
school site operation of the center w as j anticipated by February 1. 

According to preliminary (November and December) monitoring observations , 
the Birney Elementary School program was serving 113 pupils . The South 
Philadelphia High School program was serving 137 pupils . Because of the 
late funding, not all pupil -recruitment media listed in Objective 3 were used. V 
Several multi-lingual parent meetings were held atBirney to recruit pupils \ 
and meetings were held with District Superintendents and principals to inform 
them of the new program . Discussion with Arch diocesan School leadership 
of the ways that the Multi-Media Centers could be used by pupils in neighboring 
parochial schools was in progress . 



Due to the late funding ^systematic pretesting was not conducted. Posttests 
were scheduled for late Spring and analyses planned did not require pretests . 

In conclusion, 'because an indiv^ualized Milti-Media Bilingual Education 
Magnet Model was not funded by the expected date, the program was only 
partially implemented by December. Year-end findings of pupil performance will 
have to be interpreted in this light. . ; 

STATUS REPORT 

January 1977 - March 1977 / 

This report-is an\ipdate of the October- December 1976 Status . 

Report . It describes the progress made in carrying out program goals and 
enabling objectives during the period January through March 15, 1977. 

i 

By mid February , all major program goals stated in the 1976-1977 proposals 
had been accomplished. A Bilingual Individualized Multi-Media Learning Center (BIMLC) 
had been installed and was functioning at both sites. Most mufti -media ihsiruc- 
tiqnal holdings for the centers had been acquired, and computer-based instruction 
was taking place . Bilingual staff had been assigned at each site and were fulfilling 
their functions as described in the proposal. Orientation sessions had taken pl^ce 
at both centers . Monitoring showed that pupils and teachers (and some parents) fiad 
learned how to use the multi-media equipment . Daily schedules for pupils were I 
developed for group and individualized instruction in English and in the mother ' 
tongue in the classrooms and in the BIMLC . 

Project status with regarded each enabling ob/ective for 1976-1977 is described 
below: , r - < v 

Enabling- Objective 1 - Personnel described in the proposal were carrying 
put the instructional program and supportive activities heeded to operate it. (The 
October-December " 1)7^6 Status ' Report described the appointment and assignment 
of personnel . ) 

Enabling Objective 2 - Despite late funding of the program computer equip- 
ment was installed at the high school by the Division of Instructional Systems 
within one month of the date specified in the proposal. Soft-ware for computer- 
assisted and computer-managed instruction was edited , duplicated and delivered. 

Other multi-media instructional systems were acquired for the BIMLC , such as 
Systeih 80 and Language Masters. Additional instructional systems were on order. 

^ Contact had been made* through phone calls and visits to Bilingual Technical 
Assistance Centers in Miami, Florida; Carbon dale, Illinois; Falls River, Mass.; 
Washington , D.C. Seton T Jall University; and publishers in various cities in order 
to locate instructional materials in Portugese, Chinese, Korean and Vietnamese; 
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Some of these materials had been purchased, some had been lent .to the project 
and permission to reproduce them had been secured. 

Enabling Objective 3 - Because of the late funding, project management 
unable to fully i moment the "magnet" concept. The district superintendents 
and principals o^neigllb&ring public and parochial schools had been notified 
about the program. Assistance of community advisory groups" had b6Sn obtained. 
Newspapers arid radio programs had not been .used for pupil recruitment. Pro- 
gram management has set in objective describing full use of these media for 
1977-1978. 1 

.Enabling Objective 4 - Informal reading inventories for evaluation of mother- 
tongue reading performance (in languages other than Spanish) have not been 
prepared. Program management delayed undertaking this task until 1977-1978. 

Enabling Qhjectrge-& - Eight monitoring visits have shown that teachers and 
aides assigned to the Bilingual Individualized Multi -Media Center have been 
helping pupils make use of the instructional equipment and materials . Pupils 
observed had no difficulty in using the equipment and the evaluation team member 
noted that the pupils seemed enthusiastic about learning with System 80, Language 
Masters and the computer programs 



In conclusion, three of the five enabling objectives were completed by March 15. 
One of the objectives was partially fulfilled. As the one remaining objective deals 
with development of measures for evaluation , the program dan be regarded as in 
full operation. Monitoring observations show that the program is working well and 
pupils appear to be enthusiastic about the multi-media centers . 

FINAL REPORT \ 

March 1977 - June 1977 \ 



___ At y ear end, faculty at the two schools reported that 254 pupils were having 
contact with the program. At the elementaiy site, 35 spoke Korean, 25 spoke 
Portugese, and 30 spoke Spanish. These pupils were enrolled in the full bilingual 
program. Twenty-three pupils spoke other languages; for a total site enrollment 
of 113 pupils . The Korean teachers speak Vietnamese and were able to provide 
instructional support in this. language to three pupils. All pupils studied ESOL 
and w%e rostered for the Multi-Media center as part of their ESOL instruction. 

At the secondary site 33 of the 141 pupils were of Vietnamese origin, 25 were 

Chinese, 30 were Hispanic, 21 were Korean, 18 were Italian , 9 were Haitian, 
3 were French , and 2 were Greek . As the program developed , the lang- 
uages in which pupils could study changed. French-speaking pupils from 
Vietnam, Haiti and France were studying French as a first language at the 

\ . ' • • . ' - 
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beginning of the school year. When a Vietnamese faculty member (the 'site 
coordinator) was added to- the teaching staff, 13 Vietnamese pupils continue^ in a 
French class for three days per week and received instruction in Vietnamese two 
days per week . y - 



, The dominant language of pupils cannot be inferred solely from their ethnic 
origin.. Hi us, 24 pupils -, including those who continued studying Fr en ch, studied 
Vietnamese as a first language. Of thirty-three pupils studying a Chinese language, 
nine were of Vietnamese extraction . The nine Haitian and three French pupils 
studied French. All Koreans , Italians and-ffispaaifcs studied their respective 
languages . A Chinese pupil and the two Greek pupils studied English as a 
second language, but not their mother tongue . As was fh& case at the elemen- 
tary site, all pupils studied ESQL and had access to the Multi -Media Learning 
Center. 



Evaluation of product, or pupil (performance, objectives was conducted in the 
late Spring (April through June) . During this period", it was possible to gather 
at least some, information about each of these objectives. The tentative findings 
reflect the effect of the program as it was being implemented and should not 
be confused with the evaluation of a school-year-long program ! 

Product Objective 1 focused oh the rate of acquisition of English. It was 
possible to evaluate aural comprehension skills at year end, and demonstrate 
that pupils in the program scored statistically sigriificantly highfer (p< .01) than 
did similar pupils in other schools (i.e., pupils whose Sex, Age, Grade, Ethnic 
Background, Years in an English-speaking Area, and Months of ESOL were the 
same) . \ ^ - 

This finding was ba&ek on extension of the statistical regression analyses 
of Test of Aural Comprehension (TAC) scores reported-in the Appendix (see 
pages 1 5 thru 1 7) . TTiesfe analyses showed that the city-wide Title I ESOL 
program increased English Competence. 
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The Individualized Mi 
not found city-wide. It wj 
score higher than other, .s\inil 
they did or not was to ex£ 
actual scores and predictic 
performance of pupils for ^h< 
in Table 1. 



Media Magnet Model program provided-resources 
erefore possible. to ask: did its participants 
ar ESOL pupils? One way to deterimne whether 
s the residuals, or the differences between pupils 1 
from -the regression analyses. Findings about the 
m background data were complete can be found 



School i res tits were mixed . Pupils who were 
cored lower than predicted by the regression 
B scored higher^than predicted. .At Birney 
istently performed fetter than predicted . When 



At South Philadelphi 
tested with Form A of the TACls 
equation , pupils tests with 
Elementary School, pupils 

the results from both schodli Were combined, there was a trend for pupils who 
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TABLE I 



OBJECTIVE \, ANALYSIS OF TAC RESIDUALS 



Test Form 



A 



S! te 



N* Mean Res i dua 1 



N Mean ^es i dua 1 



S. Phi ladel phi a H. S 

♦ • 

Birney Elementary S 



Both Sites 



1 



22 
2*» 
46 



-0. 96 
3- 5^ 
1. 39 



19 

.8 
27 



1 . 2t 
15.35 
5. Mb 



VifS^ys is of Scores of Both Sites 



Test Fo rm 



B 



Number of Pupi Is 
Mean Residual 



Standa rd Error of 
Meart Residual 

Z "of Mean Res i dua 1 
Statistical Significance 
of Improvement 



U39 

1.21 
1.15 



27 
5. Mo 

1 .98 
2. 72 

x. 003 



P<. 15 P« _. 

Statistical significance of improvement when TAC-A and TAC-B 
results are combined: p<.01 
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took TAC A to outperform similar pupils city wide. Pupils who took TAG B were 
statistically significantly, better than predicted. * When statistical significances of 

• the findngs. of both foirns^were combined, scores were significantly above pre- 
dictions derived from TAC scores of pupils in ESOL across the city (p <.01J . 
This suggested that an individualized Multi -Media Magnet Model program enhanced 
pujpiis 1 comprehension of English, with enhancement clearest at the elementary 

t school level . . * ; 

Product Objective 2 described expected acquisition of reading skills , as 
measured by the Wide Range .Heading Test (WRRT) . fi was possible to administer 
the test in the Spiing and obtain a description of pupil performance. Overall, 
HBSults stowed that the ethnic background of the pupils £nd their ages were 
important elements in predicting pupils 1 performance. Non-ffispanic speakers 
of European languages and oyer-school-age pupils tended to out-perform other 
students . 



The WRRT involves the reading aloud of English words of increasing diffi- 
culty. It was individually administered by a researcher who attempted to 
discount the influence of accent when a word was read. Dgspite this attempt , 
WRRT scores are no doubt influenced by mastery of basic English phonology. 

.... \ \ 

- Table 2 shows WRRT scores organized by ethnic origin: Asjan , Hispanic 
and other European languages . At the elementary school, performance of the 
- three groups was similar. Grade equivalent scores averaged about the sixth 
month of the second grade. At the high school Asian and Hispanic pupils s cored 
below the non-Hispanic "Other European" group . Asians and His panics had 
mean scores eqm valeht to the middle of the 7th and the beginning of the 8th 
grade according to the published norms . S do res of pupils speaking "Other 
European" languages (and not Asians) averaged in the middle of the high school 
range. 

Figure 1 shows the relationship between Grade Level and WRRT score. 
As can be expected in a population like the one served by this program, scores 
at all grade levels were below the test maker's norms for native English-speaking 
pupils . Overall , pupils 1 grade equi valent s cores were about three-quarters 
of the scores one would expect if participants were native English speak- 
ing pupils . For example, in the eighth month of the 3rd grade , the pupils 
averaged 2. 71 , or about three-quarters of 3.8. / 

_.|_ _ _. 

Figure 2 shows the relationship between WRRT and Age. At the elementary 
school there was a general upward trend (the peaks and dips are to be expected 
because of the relatively small nuriiber of pupils in most age groups) . At the 
high school the pattern was quite different. Youngest and oldest pupils had high 
scores . Pupils of common high school age (sixteen! throiigh eighteen years old) 
tended to have grade equivalent scores in the 7th and 8th grade range. 



TABLE 2 



WIDE-RANGE READING TEST GRADE EQUIVALENT SCORES 
CATEGORIZED BY ETHH I C ORIGIN OF PUPILS 
AND BY SITE" 





















intary 


Schoo 1 


Hi 


gh School 




Ethnic Group 


N 


Mean 


SD 


N 


Mean 


SD 


Asian 


39 


2.77 


1.50 


46 


7.57 


3.18 


Hispanic ; 


no 


2.53 


1 . 30 


17 


• 8.27 


3-59 


Other European 


24 


2.43 


1.^5 


20 


1 1 .03 


3.05 


A1 1 Groups * / v 

. — \ 


. 103 , 


2. 59 


1 . 42 


83. 


8. 55 


3. 54 
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Product Objective 3 desciibed gains of Hispanic pttpils 1 competence ih^-ead- 
ing Spanish. Although during the Spring 1977, pupils were tested, the number 
was too smpll to draw any firm conclusion. 

At the elementary school site, pupils in Grades 1-3 were tested with Test de 
destrezas basicas en lectura. The one first-grade pupil scored at the 95th 
percentile; tfrtTaverage score of the tljree second-grade pupils was equivalent to 
the 50th pe/centil e : an d^he 12 third-grade pupilis 1 average score was equivalent 
to the 20th percentile when compared with rural Puerto Rican end-of-year norms. 



• Pupils In Grades 4 and/6 were tested with Prueba de < lectura (de Puerto Jt ico) . 
Th - average score of the ten 4th -grade pupils who were examined ^ith the com- 
plete test Was eqvdvalenj/to the 10th percentile of rural Puerto Rican endTof-year 
norms-. The tW4ttfa^rade pupils 1 average score was equivalent to the 5th per- 
centile on these norms . • 

The pattern of test score observed at the elementary school was'sinilar to 
the one found in the Let's Be Amigos project, which was begun in 1959. Pupils 
in lower grade levels had average scores at or above Puerto Rican norms . Pupils 
in higher grade levels averaged below these norms . (See Offehberg et. al. The 
Title VII Project Let's Be Amigos: Evaluation^of Ae Sixt h Year , 1974- 1975 . 
School District of Philadelphia, 1976, pgs . 26-34) . 

_* _ ' > 

At t&e high -school level, pupils were tested with the Prueba de Lectura 
Inter- americana (L4, CEs) . This test was designed for junior high school pupils , 
but previous experience with Hispanic pupils in bilingual programs suggested 
it would be more reliable than the level designed for high school . A table of 
equivalent scores of adjacent test levels was used to obtain pd^entile ranks 
appropriate for high-school grades. (Manual of Tests of Genial Ability and 
Teats^i Beading, Inter-American Series, Forms CE and CEs , DE and BEs . , 
Austin, Texas: Guidance Testing Associates , 1967.) 

The seven lOth-grade pupils averaged 36. 8,. equivalent to a fail island-^ 
wide Puerto Rican percentile rank of 32.8. The ten 11th -grade pupils 
averaged 38.5, equivalent to a fall percentile rank of 22.9. (There were no 
tests of twelth- grade pupils) • 

The high school percentile ranks are substantially above those obtained 
when Hispanic high school pupils of the Let's Be Amigos program were tested. 
(See Offenberg Op. Cit. p. 45-51) . 

Product Objective 4 described an anticipated reduction of the drop-out 
incidence in the high school component. Eighty-two pupils were on roll in ESOL 
and/or were receiving instruction in their mother tongue in October 1976. Of 
these, five left school during the school year — one moved, one was committed 
to a correctional institution , and three (3.8%) dropped out of school . 
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The proposal called fbr comparison of these data with fee drop-out rate 
anbng similar pupils who attended the same school the year before theyprogram 
was implemented. This was not possible becaiise the pupils identified by school 
personnel during the base-line year did not appear to. come from a comparable 
group . 

Product Objective 5 described anticipated changes in high school pupils 1 
attendance. Among pupils who attended South Phttad^phia High School in 
both 1975-1976 and in 1976-19 77 no significant change in attendance was observed. 
In 1976-\19 77 nine Hispanic pupils in this group were present an average of 80.1% 
of the da^s, 17 Asian pupils averaged 93.6%', and 14 speakers of European languages 
other than Spanish averaged 86 .0%. A sign test was computed to compare each • 
pupil's attendance with that of his/her previous year. The computations yielded 
a r of .etjTvhich was nonsignificant. 

• PupilsNwho attended the high school for the first time in 1976-1977 had poorer 
attendance records than did the pupils who had attended the school for two years . 
The 15 Hispanic pupils in this group averaged 77 .6 % p resent , the 48 Asians 
averaged 86.0%, and the 10 speakers of other European languages averaged 63.8%. 
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Pupil performance data reflect one-halPyear of pre-program instruction, 
and one- half year of the Individualized Multi -Media Bilingual program . Even 
so, aural comprehension scores were significantly higher than expected at 
the two target schools . 

Findings in other areas may be useful as a context , in which the results of 
1977-1978 may be interpreted. Testing of reading in English suggested that it. 
would be adivisabie to use another instrument in addition to, or instead of, the 
WRRT for summative evaluation. As the Wide Range Reading Test measures decod- 
ing and word recognition, it is possible that the scores reported are inflated . If 
the pupils were examined with an instrument demanding comprehension, the scores 
might be lower . Evaluators had this impression during the testing. 

Despite this concern , the WRRT provided information which can be used by 
project management. At the secondary level , it revealed that youngest and 
over-age pupils h ad high* scores , and that the ethnic origin of the pupil is 
related to score . This suggested that there were subgroups of pupils 
within the program with heterogeneous needs . The BIMLC will probably be 
especially valuable at the secondary level because it accomodates heterogeneous 
pupil populations easily. " • 

•& the. course of testing Hispanic pupils in the mother torlgue, it became 
evident that, especially at the elementary school site, the number of pupils in a 
X trade who spoke this language was too small to yield stable program assessment 
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data, tfhen the number of pupils in a grade is low, as the number of elementary 
school Hispanics often was , testing in the first language may be useful forthe _ 
individual pupil , but statemeilts about ,prdgram effectiveness are lisky . With the 
risk acknowledged, it appears that the pattern at the elementary school site was • 
similar to the pattern observed in the 'older , Let's Be Amigos project— perform- 
ance was as good as or better than , Puerto Hi can norms in the lower grades , 
but declined when compared to these norms in higher grades of elementary school . 

• fix addition to their linguistic diversity, high -school pupils in this program- 
appear to differ in at least two ways from the bilingual high school program which 
was established with Title VII funds as part of the Let f s Be Amigos Projeqt. First,. 
Hispanic pupils read Spanish better than did pupils in the last evaluation of that 
project. Second, preliminary evidence gathered about drop -but incidence, . 

suggested that high-school program participants were veryunlikely to leave x 

school. The dropout rate of 3 .85% is markedly lower thap/fhe 1976-1977 city-Vide 
high-school rate (21.2%), the rate for all pupils at^sndirfg South Philadelphia \ 
High School (26.7%) , and the rate for Hispanic pupils attending the school (24.0%) . 
This suggested that the high school program may not have recruited all pupils 
who might have benefitted . During the coming years , full implementation of the 
pupil recruitment objective should be given special attention in order to assure that 
there are no other pupils who should participate in the program . 

In conclusion. An Individualized Multi -Media Bilingual Program appears to 
have had as successful a first year as possible, considering the hurdle of late 
fun,ding. There is evidence that pupils performed' well, stayed in school, 
and, especially at the elementary school site, pupils had aural comprehension 
levels above those enrolled in ESOL in other educational settings . 
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APPENDIX 
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EVALUATION OF THE AURAL COMPREHENSION 
OBJECTIVE OF THE CITY WIDE 
TITLE I ENGLISH TO 
SPEAKERS OF OTHER 
LANGUAGES PROGRAM 



OBJECTIVE / 

The longer a pupil studies ESOL, the higher he/she 
will score on the Test of Aural Comprehension of English (TAC\ 
when the pupil's background is taken into account. A. regression 
equation computed on tests administered in March, 1977 will show 
a statistically significant relation (p<.10) between the length 
of time a pupil studied ESOL ■ and TAC scores when sex, age, grade, 
place of birth, mother tongue and length of residence on the 
mainland are controlled. 

_ _ « 

This objective was attained. 

The TAC was administered In March and April In Grades 
1-12 at all participating schools. A total of 1 ,503 pup! 1 s were 
tested with either form of \he test (A or B) . Analysis of the 
867 pupils with complete Information Is based on the general linear 
model and Indicates a positive relation between test scores and 
length of time in an ESOL/bi i ingual program. This relationship 
was statistically more significant than the specified .10 level. 
The resul ts suggest that aural comprehens i on tends to Increase 
with continued participation in ESOL to an extent beyond that_ 
which coul d be expected from the pupils' backgrounds and their 
living in an Engl ish-speaking environment without studying ESOL. 
The resul ts of the regressi on analysis of Form. A are shown In 
Tabl e 1, of Form B in Tabl e 2. 

\ The variabl e sex was held stat I st I cal 1 y constant to 

control for d i f ferent rates of growth In language acquisition 
between bo/s and girls and for possible biases that might resul t 
frofi some Schools serving only boys or girls. Table 1 shows that ■■■ 
for\the model posited, the relation between sex (coded 0 for_majes - 
and \l for females) and score oh Form A Is not s I gn I/f i cant . Table 
2 confirms this finding in the analysis of Form B. / This may be 
Interpreted as Ind Icatlng that for the skll Is covered by the 
there is no difference In score attributable to sex. I t may a o 
be Interpreted as showing the TAC Is. equally fair to both sexe 

Since TAC scores probabl y "ref 1 ect skll 1 s acquired In 

school , grade was held constant. Tables 1 and 2 Indicate that 

for both forms of. the TAC a pupil Is 1 ikely to know 2.6 Items more 
than an equivalent pupi 1 who is one grade lower. As expected, this 
Impact of grade Is highly significant. t : 

■ H j , ■> , • •;• \ •<;•.. . •. : .... ' ... 
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The age of the pupil was not s i gn i f I can 1 1 y related. to 
score when other variables were taken into account ; The high 
col ] i near i ty between age and grade (their correlation is about 
,97) makes i h terpreta t-l on difficult. 

The ethnic background of . the pupil was also considered 
in the analysis. Pupils whose mother tongue is Spanish were 
coded as 0 and all other language groups were coded as 1 . Where 
Information about mother tongue was mfssihg, place of birth 
(Puerto Rico or other) was used. Controlling for this va r i able, 
it was thought, would reduce the influence of social and ethnic 
characteristics on program evaluation. The two tes/t forms yielded 
Inconsistent results, both for the direction of the. relationship . 
and Its significance. Hispanic pupils did significantly better on 
Forr A but on Form B there was no significant difference between 
the two ethnic groups. This suggests that further inquiry is 
necessary. 



The acquisition of 1 anguage skills would seem to be 
related to the amount of exposure to English outside of the program. 
One measure of this construct is the time in the continental United 
States. Table 1 indicates that an additional "year irt an English- 
speaking area increases scores on TAC-A by Items and Table 2 
shows the increase to be a bit less than one item. 

The key question is whether participation in an ESOb 
program contribute? to aural comprehension over_and beyond the 
contributions of grade, age, sex , fethn i c background and number of 
years on the mainland. Tables 1 and 2 show a significant positive 
relation between months in ESOL (based on 1 0-month years) a^nd test 
scores, independent of the other variables. 

The results in Tables 1 and 2 predict that an additional 
month on the mainland when the pupil is not attending school will 
Increase his score by .036 on TAC-A and .032 on TAC-B. 1 f the _ 
pupil spends this month in school wi thout -ari 1 ESOL program the im- . 
provement will rise to .301 and .212. If' the pupil is participating 
In an ESOL program the improvement will r i se tc 1 . U 62 and . *i 1 5 . „ - 
Thus we conclude that there is an increase of more than four points 
for the average pupil over the 1 0-mdn th academi c year. Since the 
average score is 18 .9 for pup i 1 s taking TAC-A in ttfe spring and 21.3 
for TAC-B , these gains are considered substantial. 
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TABLE I 

REGRESSION OF TAC-A SCORE ON VARIOUS ESOL PUPIL CHARACTERISTICS 



Independent Van able 


d we i §n t 


S t an? f I can ce 


Sex 


".235 


.776 


Grade 


2. 646 


.0001 


Age 


-.706 


.1217 . \ 


Ethnic Background 


-1 .65 


. 0986 i 


Years in Engl i sh- speaking Area 


1 . 1 A 3 


.0001 


Months in Program 


.161 


.0061 



Based on 460 cases for whom a 1 1 relevant information was 
a va liable. 

Mean score = 18. 966, S.D.=8.732, multiple R2=.i»5*» 



TABLE 2 

// 

, REGRESSION OF TAC-R SCORE ON VARIOUS ESOL PUPIL CHARACTERISTICS 



I ndependent Va r i able — 


b Weight 


S i g n i f tca^nc e 


Sex 


. .682 


- // 
.5137 / 


Grade 


2.59* 


.0,003 


Age 


-.508 


.^283 


Ethnic Background 


.Okh 


.973* ., 


Years in English-speaking Area 


.895 


.0001 / 


Months in. Program 


.123 


.022^ 

/ 


Based on 407 cases for Whom all 


re 1 cvant 


/ 

Information was -j 



aval 1 abl e. 

Mean score '«= 21 .333 , S.D. « 1 0. 306 , mul 1 1 pi e R2-.368 



